Deferred session requests #256

toji · 2017-07-11T05:08:00Z

This is an update to how we would handle device activation events, which I alluded to in #194. I've also combined it with some of the navigation concepts from #247, which appears to be another magical GitHub PR closure. (I'm doing something weird with my branches, I guess.)

The intent behind this style of session request is to make UA transition logic more predictable. By making a deferred request the UA knows that it can switch to presentation mode when the criteria is fulfilled, so there's no bouncing back and forth into Javascript and hoping that the page calls requestSession in response to an event firing. It's definitely something we've seen would be beneficial in Chrome.

I have some concerns out the gate that I'd like others opinions on: Not sure if the terminology (deferred session request, deferTill, etc) is right for the functionality. It especially feels a bit odd in the "navigate" case. Also it's awkward to me that activate is this new deferred thingy while deactivate is still a normal event. Finally, I kinda wonder whether or not we need getNavigationDevice. You could technically get by with just iterating through all of the devices and requesting a deferred navigate session against them. It would reject immediately on anything that wasn't valid for navigation. That feels a little ugly, but in reality most of the time you'd only have one device anyway. So maybe that's good enough for V1?

So I'd love feedback on any of that, or anything else that you've got thoughts on. Thanks!

Preview | Diff

toji · 2017-07-11T18:59:19Z

explainer.md

@@ -478,60 +478,59 @@ function onDrawFrame() {
 }
 ```

-### Presenting automatically when the user interacts with the headset
+### Responding to a reset pose


Sorry, I moved the automatic presentation section below this one to help it flow better with the navigation section, but it's made the diff kind of ugly to read. The entire "Responding to a reset pose" section is unchanged.

toji · 2017-07-11T22:32:47Z

Went over this proposal at length today on the implemetors call. A few things that were brought up:

There are some scenarios where things like hardware proximity sensors are triggered but it's not appropriate to create the deferred session anyway. (Microsoft pointed out that in many cases the user putting on the headset may want to go to an OS shell rather than jump into WebVR immediately.) To this end we should be clear that the decision of when to trigger this is in the UAs hands and should only happen when there is a very clear intent that can be derived from the user's action.
In addition to that, we almost certainly need permissions around this feature. (Might be appropriate for a follow up PR). Something like a permission pop up stating "example.com wants to automatically enter VR. [Allow Once] [Always Allow] [Deny]"
It may be worth investigating if there's a more appropriate name that can be used than "activate"? We want to drive home the idea that pages want to register for this in most cases.
Should we consider allowing pages to register for multiple criteria at once? { deferTill: ['activate', 'navigate'] } If so we need to specify how promise rejection is handled and indicate how.

I may have missed logging some of the subjects we covered, but I think those were the major points.

@NellWaliczek

Requested by @NellWaliczek. This is for the sake of clarifying things for TAG review if we don’t get a better solution in place before they look at the explainer, since we know this ISN’T the right way to handle this. Ideally we’ll put it back in with #256 or a similar PR.

@NellWaliczek

Requested by @NellWaliczek. This is for the sake of clarifying things for TAG review if we don’t get a better solution in place before they look at the explainer, since we know this ISN’T the right way to handle this. Ideally we’ll put it back in with #256 or a similar PR.

@NellWaliczek

…iner Requested by @NellWaliczek. This is for the sake of clarifying things for TAG review if we don’t get a better solution in place before they look at the explainer, since we know this ISN’T the right way to handle this. Ideally we’ll put it back in with #256 or a similar PR.

RafaelCintron · 2017-07-27T02:46:32Z

With this new promise approach, I think not being able to unsubscribe to the deferral for activation is a step backwards over having a plain old event handler.

On Facebook, I typically have a steady stream of 360 videos in my feed due to the people I follow. Clicking on one of these videos renders a larger version over the top of the feed. If would be great if, in this mode, putting your phone in a headset triggered automatic WebVR. However, Facebook wouldn’t want automatic WebVR if there is more than one video on the page and certainly not if there are zero videos on the page. The same can be said for real estate websites or other “progressive enhancement” pages where automatic WebVR is not always the right thing to do. I think we need to provide more flexibility than what is proposed. Cancelling the deferral by calling endSession on promise resolution would likely entail more system bringup/teardown and is awkward from an API perspective.

toji · 2017-07-27T05:23:00Z

Thanks for the comment. I agree that the lack of a method to cancel the handler is problematic, and I appreciate you bringing it up since I hadn't considered it before. I'd like to talk over some options with the developers on my end that were requesting a change in the first place, but I'd appreciate suggestions of alternative APIs in the meantime! For the moment it seems like going back to an event handler is the least problematic route.

RafaelCintron · 2017-07-27T21:15:02Z

I agree event handlers seem like the best option in lieu of the current proposal. However, I would like to hear what @NellWaliczek and her colleagues discuss early next week on the topic.

@NellWaliczek

…iner Requested by @NellWaliczek. This is for the sake of clarifying things for TAG review if we don’t get a better solution in place before they look at the explainer, since we know this ISN’T the right way to handle this. Ideally we’ll put it back in with #256 or a similar PR.

toji · 2017-09-26T17:52:50Z

Resurrecting this thread following a discussion I had with @RafaelCintron at a Khronos face to face last week. I had been doing some research on this and bounced an idea off of him, and he responded with a simpler idea. I think both ideas have merit and both have tradeoffs, so I wanted to outline them here for discussion:

Brandon's Proposal

I was reading about the direction that the fetch API was taking for handling cancelable promises (which, it turns out, is a long and drama filled read). They have eventually settled on a concept of an AbortController which is then extended into a FetchController for use with the fetch function. This feels a bit complicated at first glance, but I feel like I can see how it was arrived at.

Anyway, I could see this pattern being successfully applied to the concept of deferred sessions in a very similar manner:

let controller = new VRSessionRequestController();
controller.deferUntilActivate = true; // Kinda don't care what the syntax is here.

vrDevice.requestSession({ exclusive: true, signal: controller.signal })
  .then( /*...*/ )
  .catch( (err) => {
    // If the request is aborted you'll end up with with err being an AbortError.
  });

function onSomeCriteria() {
  // Oh noes! Some criteria has been fulfilled and now it is no longer appropriate
  // to start presenting on activate. Better cancel the request!
  controller.abort();
};

Pros:

Allows a lot of fine grained control by the developer over outstanding request lifetimes.
Adheres to an established pattern, though it's still new and is only specified in the whatwg spec.
Enables better library isolation, since they can prevent "spooky cancelations at a distance."

Cons:

Somewhat complicated and non-intuitive, especially given that the pattern is new.
More IDL needed.
Depending on how we approach it may introduce dependency on AbortController, which isn't in any browsers yet.

Rafael's Proposal

Upon me outlining this idea, Rafael proposed a more straightforward but possibly more limiting route: Have a single cancelSessionRequests method on VRDevice that would cancel any and all outstanding session requests, and then handle the deferrals as has been previously proposed by this pull request:

vrDevice.requestSession({ exclusive: true, deferTill: 'activate' })
  .then( /*...*/ )
  .catch( (err) => {
    // Again, canceling before the session is fulfilled lands you here.
  });

vrDevice.requestSession({ exclusive: true, deferTill: 'navigate' }).then( /*...*/ );

function onSomeCriteria() {
  // Oh noes! Some criteria has been fulfilled and now it is no longer appropriate
  // to start presenting on activate/navigate. Better cancel the request(s)!
  vrDevice.cancelSessionRequests();
};

Additionally he suggested that under this system multiple requests with the same creation criteria could return the same promise for each call. This is indeed simpler: it's only one additional function instead of a couple of new interfaces. It does lack some fine grained controls, though. A library that puts out a deferred session request may randomly have it cancelled by a developer who was unaware that it was even requested and instead was trying to manage their own session request lifetime.

Pros:

Simpler API
More intuitive to call
No dependencies on other interfaces/specs

Cons:

Less intuitive behavior when mixing multiple libraries.
"Spooky cancelations at a distance."
Less extensible in the future.

Anyway, as I said I think both variants have merit so I'd love some additional opinions on which of these approaches you would lean towards (or suggestions for a different approach all together!)

kearwood · 2017-09-26T20:59:04Z

The 2nd, "Rafael's Proposal", feels better to me, assuming that everyone is comfortable with pushing library interaction up higher into the stack

toji · 2017-09-29T18:13:06Z

David mentioned at one point that if the abort controller is the way the web platform is going we should follow it. Wanted to point out that that does seem to be the case (exhibit A).

cvan · 2017-10-24T18:19:59Z

@toji know this is a bit old, but do you need this reviewed, or should I wait on updates?

toji · 2017-11-01T19:22:39Z

Following up from conversation on the most recent call, I've refactored this change to account for the proposed transition to an AbortController-based interface. (Annoyingly I had to squash the history out of existence due to a really wonky merge that was spamming this PR with a bunch of unrelated changes.) So, yes @cvan, I could use a review about now. 😁

One outstanding question for me that this change doesn't address (and which I think is a matter of platform expectations.) If a developer creates a standard AbortController and passes it's signal into requestSession should we accept it and abort on demand like normal? (That's probably nonsensical in most cases, as we don't expect a normal session request to be outstanding for more than a few milliseconds most of the time.) Or should we only accept VRSessionRequestController instances? I'll poke around and see if I can find someone in the know to ask about that.

Anyway, comments on that issue or the rest of the change welcome! Tried to be fairly explicit in the text about various edge cases but that can lead to dense and hard to parse prose so please let me know if something is confusing.

toji · 2017-11-02T17:33:48Z

I reached out to @jakearchibald about this, given that he was one of the developers involved with the creation of AbortController and he had this to say:

waitForActivate and waitForNavigate feel like options rather than actions, so it feels like they should sit on the options object rather than the controller/signal.

const controller = new AbortController();
const signal = controller.signal;

requestSession({
  waitUntilActivate: true,
  signal
}).then(() => {
  // …
});

This means you don't need to extend the AbortController/Signal – not that extending it is a bad thing, but it doesn't seem necessary here.

Which, now that it's been pointed out, seems really obvious to me and I'm not sure why I didn't take that approach in the first place. 😓 It's definitely simpler in terms of IDL within our spec. I'll update the PR to reflect that momentarily.

toji · 2017-11-02T18:01:55Z

PR is now updated to account for the above change.

cvan

this looks great. my feedback is mostly on minor typos, questions about gaps in description of intent and lifecycle.

I’m a big fan of the approach you’ve taken here. thanks for incorporating everyone’s feedback here. this is one of the last big changes to getting first-class VR navigation and UX 👍

cvan · 2017-11-03T02:26:43Z

explainer.md

@@ -591,6 +591,90 @@ vrSession.addEventListener('resetpose', vrSessionEvent => {
 });
 ```

+### Presenting automatically when the user interacts with the headset
+
+Many VR devices have some way of detecting when the user has put the headset on or is otherwise trying to use the hardware. For example: an Oculus Rift or Vive have proximity sensors that indicate when the headset is being worn. And a Daydream device uses NFC tags to inform the phone when it's been placed in a headset. This is referred to as the `VRDevice` being "activated", and depending on context it may represent the user showing a clear intent to begin using VR. Many WebVR application may want to begin presenting automatically in these scenarios.


first sentence could be simplified to Many VR devices detect when the headset is being worn or used by the user.

change Many to A (or use plural applications)

cvan · 2017-11-03T02:27:57Z

explainer.md

+
+Many VR devices have some way of detecting when the user has put the headset on or is otherwise trying to use the hardware. For example: an Oculus Rift or Vive have proximity sensors that indicate when the headset is being worn. And a Daydream device uses NFC tags to inform the phone when it's been placed in a headset. This is referred to as the `VRDevice` being "activated", and depending on context it may represent the user showing a clear intent to begin using VR. Many WebVR application may want to begin presenting automatically in these scenarios.
+
+In order to start presenting when the `VRDevice` is activated pages can request a deferred session. To request a deferred session the option `{ waitForActivate: true }` is passed into the `requestSession` call. When a session is requested with `waitForActivate` the request will remain outstanding until the `VRDevice` is activated, at which point the promise will resolve with the requested session (or reject, if necessary.)


can you add a comma after activated?

outstanding -> pending?

.) -> ).

cvan · 2017-11-03T02:31:37Z

explainer.md

+
+In order to start presenting when the `VRDevice` is activated pages can request a deferred session. To request a deferred session the option `{ waitForActivate: true }` is passed into the `requestSession` call. When a session is requested with `waitForActivate` the request will remain outstanding until the `VRDevice` is activated, at which point the promise will resolve with the requested session (or reject, if necessary.)
+
+Deferred sessions must be exclusive and will not be fulfilled if there is already an exclusive session active for the VR hardware device when the activation action occurs. Deferred requests do not need to be made within a user gesture.


what happens if an initial activation and deactivation happen before the requestSession call is made (when {waitForActivate: true})?

My intent would be that the activation is ignored and we would wait till the next one to resolve the session. We could also allow it to pick up an activation that has happened a couple of seconds prior to the request being made, but I'm not sure if that's going to be a commonly encountered scenario.

cvan · 2017-11-03T02:32:06Z

explainer.md

+vrDevice.requestSession({ exclusive: true, waitForActivate: true }).then(OnSessionStarted);
+```
+
+Once the deferred session request has been fulfilled or rejected a new deferred session request will need to be issued if the page wishes to respond to future activation actions.


can you add a comma after rejected?

this is kind of in line with my question above, but doesn’t cover that exact case of activate -> deactivate prior to

cvan · 2017-11-03T02:33:39Z

explainer.md

+
+Once the deferred session request has been fulfilled or rejected a new deferred session request will need to be issued if the page wishes to respond to future activation actions.
+
+To detect when the user removes the headset, at which point the page may want to end the session, listen for the `deactivate` event. Note that not all devices capable of detecting activation can reliable detect deactivation, so pages are not guaranteed to receive an associated deactivate event for each activation action.


reliable -> reliably

I’d add backtick code marks for deactivate

cvan · 2017-11-03T04:06:41Z

explainer.md

+    waitForActivate: true,
+    signal: abortController.signal })
+  .then(OnSessionStarted)
+  .catch((reason) => {


out of curiosity, would the AbortController-backed signal always throw or reject?

can you remove the parentheses around the argument

Um... not entirely sure what you're asking about the throw/reject. The abort signal causes the promise to reject when it's controller's abort method is called if it has not already resolved.

cvan · 2017-11-03T04:07:52Z

explainer.md

+    if (abortController.signal.aborted) {
+      console.log("Session request was canceled by the page.");
+    } else {
+      console.log("Session request was reject by the system: " + reason);


I would do one of the following:

“, reason);

“, reason.message);

cvan · 2017-11-03T04:09:25Z

explainer.md

+  .catch((reason) => {
+    if (abortController.signal.aborted) {
+      console.log("Session request was canceled by the page.");
+    } else {


nit: this conditional logic doesn’t necessarily mean the “system” rejected it. there could’ve been an error thrown in OnSessionStarted.

cvan · 2017-11-03T04:12:16Z

explainer.md

+
+```js
+// Requests that a session be created when the device is activated.
+let abortController = new AbortController();


it might be nice to show another example to show that you can listen for an event as well:

abortController.addEventListener('abort', evt => { console.log('Aborted'); });

cvan · 2017-11-03T04:12:46Z

explainer.md

+vrDevice.requestSession({
+    exclusive: true,
+    waitForActivate: true,
+    signal: abortController.signal })


feel free to put the brace on the next line, for clarity

toji · 2017-11-03T21:58:55Z

Addressed your comments, and had one outstanding question (repeated here since GH collapsed the comment): Is there a downside to moving the deactivate event to VRSession? It seems like it only really applies to exclusive sessions, we probably don't want to relay that information on when the page isn't actively using the headset, and it would allow us to drop the VRDisplayEvent and VRDisplayEventInit types entirely.

cvan · 2017-11-03T22:35:55Z

I actually like that a lot. it cleans up the interfaces. my only concern would be that developers probably don’t want to manage different Promises for the VR Sessions. it’s nice to always have one place to add your event listener. I can’t think of a common use case where this new change would be cumbersome. 👍

toji · 2017-11-04T17:55:37Z

Okay then, done! I'm feeling pretty good about the general state of this feature but I'm waiting to merge because @NellWaliczek let me know that Microsoft had some feedback on this change incoming.

toji · 2017-11-11T05:22:55Z

Talked with @NellWaliczek about this extensively at TPAC this week, which helped me better understand Microsoft's position on this. I'll recap a bit of it here prior to updating the PR for context.

One big Microsoft is concerned about the concept of "activation" being potentially being triggered when inappropriate. For example: When you have a couple of browser windows with some WebVR content on them but they're not necessarily what the user is focused on at the moment. Should putting on the headset trigger those pages, or the default system "home" environment? If you do launch VR content which of the browser pages takes priority? Will that be consistent across browsers and platforms? It's something that may make sense on some devices that are more modal with VERY strong VR entry signals (placing and Android device into a viewer), but not platforms that are more multitasking friendly.

Which leads to a secondary concern of activate signals being applied unevenly, which may lead developers to make bad platform-wide assumptions. For example: We've already seen at least one example of a fairly high-profile site that was inaccessible from Edge because they relied solely on "activate" events to trigger VR instead of providing a button on the page. This is definitely a behavior we don't want to encourage.

For my part, I still think that the concept of activate has merit in a couple of different areas:

It's proven to be a useful pattern on Daydream-style setups
It works well in a kiosk/demo environment where users stepping up to a station and donning a headset can trigger the VR content.
It could enable things like UX affordances to advertise that a page is VR-capable and offer easy ways to launch the content.

But given Microsoft's concerns I'm willing to defer (😏) that part of the deferred sessions proposal till we can have some further discussion.

Navigation, on the other hand, seems to be a universally agreed upon "good idea", so any concerns there come down to behavioral details. I think that navigate as it's been presented here should work out just fine, but Nell and talked about a few extended uses and considerations for it that would be good to capture here (and potentially in the explainer as well, where appropriate.)

For one, UAs could enable "Deep Links" to pages which start up immediately in WebVR mode with the proposed navigation pattern. It would work like so: Page attempts to request a navigation session but it's rejected immediately because the criteria for navigation hasn't been satisfied. Nevertheless, the browser can remember that the page did make the attempt and when the user goes to create a bookmark or add to homescreen or what have you they can be given an additional option in the dialog:

Launch page in VR?

Given that's UA behavior we can't really spec it, but maybe some non-normative text is in order.

Another interesting option would be the ability to have links on a 2D page be given some markup that indicates that they want to launch the linked page in VR if available. It would be useful for content galleries that want to have a 2D page tying together multiple VR experiences. This may look something like:

<a href="/webvr_example_1.html" target="_vr">View Example 1</a>

The UA of course would have a lot of latitude around how to handle that. If the system doesn't have VR capabilities then it would just function as a normal link, or it could choose not to honor the request if the user has previously denied the origin the ability to enter VR automatically. The UA could also give visual indicators that the link will jump into VR if needed, such as a different mouse cursor upon hover.

Anyway, that's a slice of the conversation we had, there's a few other tweaks that I'll try and work into the explainer text too. That'll likely be incoming beginning of next week.

toji · 2017-11-13T23:17:43Z

Okay, for the time being I've stripped the concepts of activate and deactivate out of the explainer, and along with them the necessity for the AbortController. I've got it all stashed away for a later PR, though. 😉

Reworked the text to be navigation-centric, but I've left out the previously described link target behavior and non-normative text about deep links because I feel that's better left to a follow-up PR as well, where the behavior can be discussed in a more focused way. Please take a look at the current text and let me know if there's anything that should be fixed prior to merging!

NellWaliczek

Thanks for accommodating my request to focus this PR on just the navigation scenarios! I've added a few comments that may have gotten lost in the cracks of our conversation in person last week :)

NellWaliczek · 2017-11-13T23:43:26Z

explainer.md

+
+WebVR applications can, like any web page, link to other pages. In the context of an exclusive `VRSession`, this is handled by setting `window.location` to the desired URL when the user performs some action. If the page being linked to is not VR-capable the user will either have to remove the VR device to view it or the page could be shown as a 2D page in a VR browser.
+
+If the page being navigated to is VR capable, however, it's frequently desirable to allow the user to immediately create an exclusive `VRSession` for that page as well, so that the user feels as though they are navigating through a single continuous VR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "VR navigable", which activates a couple of new behaviors to allow for the desired navigation experience. UAs may also place additional restrictions on what conditions are considered VR navigable, such as restricting it to only same-origin URLs.


Not that it is a guarantee that developers will pay attention to this, but can we add language indicating that it's important for pages to have an "Enter VR" button even when also requesting the "on navigate" session?

NellWaliczek · 2017-11-13T23:45:07Z

explainer.md

+
+WebVR applications can, like any web page, link to other pages. In the context of an exclusive `VRSession`, this is handled by setting `window.location` to the desired URL when the user performs some action. If the page being linked to is not VR-capable the user will either have to remove the VR device to view it or the page could be shown as a 2D page in a VR browser.
+
+If the page being navigated to is VR capable, however, it's frequently desirable to allow the user to immediately create an exclusive `VRSession` for that page as well, so that the user feels as though they are navigating through a single continuous VR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "VR navigable", which activates a couple of new behaviors to allow for the desired navigation experience. UAs may also place additional restrictions on what conditions are considered VR navigable, such as restricting it to only same-origin URLs.


This flags the newly loaded page as "VR navigable", which activates a couple of new behaviors to allow for the desired navigation experience.

This phrasing doesn't read super clearly to me...

NellWaliczek · 2017-11-13T23:48:43Z

explainer.md

+
+If the page being navigated to is VR capable, however, it's frequently desirable to allow the user to immediately create an exclusive `VRSession` for that page as well, so that the user feels as though they are navigating through a single continuous VR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "VR navigable", which activates a couple of new behaviors to allow for the desired navigation experience. UAs may also place additional restrictions on what conditions are considered VR navigable, such as restricting it to only same-origin URLs.
+
+In order to start presenting automatically when a page is VR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is VR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not VR navigable the request will reject immediately.


Is it really "deferred"? Or can we simply just remove the requirement for the session initiation to be performed via a user gesture if invoked early enough in the new page's lifetime? If so, what is "early enough"?

NellWaliczek · 2017-11-13T23:50:33Z

explainer.md

+
+If the page being navigated to is VR capable, however, it's frequently desirable to allow the user to immediately create an exclusive `VRSession` for that page as well, so that the user feels as though they are navigating through a single continuous VR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "VR navigable", which activates a couple of new behaviors to allow for the desired navigation experience. UAs may also place additional restrictions on what conditions are considered VR navigable, such as restricting it to only same-origin URLs.
+
+In order to start presenting automatically when a page is VR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is VR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not VR navigable the request will reject immediately.


Can we be more specific about when in the page lifetime the promise will be expected to fulfill? Will the DOM already be fully set up (for grabbing a canvas object for example)? Or do we want this to be earlier so fewer resources need to be previously spun up?

NellWaliczek · 2017-11-13T23:51:25Z

explainer.md

+vrDevice.requestSession({ exclusive: true, waitForNavigate: true }).then(OnSessionStarted);
+```
+
+> **Non-normative Note:** The UA should provide a visual transition between the two pages. At minimum the previous page should fade to black and the new one should fade in from black. Additionally, if the UA cannot display pages in VR it is recommended that it should instruct the user to remove the headset once the UA switches to displaying a 2D page.


At minimum the previous page should fade to black and the new one should fade in from black.

Too prescriptive :) What if the user agent wants to fade to white?

NellWaliczek · 2017-11-13T23:53:18Z

explainer.md

@@ -652,6 +670,7 @@ partial interface Navigator {

 dictionary VRSessionCreationOptions {
  boolean exclusive = false;
+  boolean waitForNavigate = false;


I'm not clear on why this parameter would actually be necessary. I get that it might be left over from the activation design, but I'm not see the value it's adding given the current design... am I missing something totally obvious? ^_^

Well, for one I want to leave the door open for activation and I think this establishes the right pattern for it. (I know you'd rather not have activation be a thing, but I'm not giving up on it yet.)

Secondly, I'm not really comfortable with the idea of "Make a request that would otherwise fail but if you do it within this special timeframe it might sometimes succeed. Maybe." In my opinion the API should be more explicit in it's intent than that. It would feel less weird to me if we said developers could add a listener for a vrnavigateready event and you can make session requests within that callback, but that's the model that WebVR 1.1 uses which we are explicitly trying to get away from because adding the event listener is not a strong guarantee for the browser that session creation will happen.

I hear what you're saying about being more clear about the time span that this call can validly be made in, and I'll update the doc to account for it.

toji · 2017-11-14T19:12:14Z

Okay, made some more updates based on Nell's feedback, but I've kept the need for an explicit waitForNavigate flag and the concept of deferred sessions with it. I'd appreciate feedback from other vendors to hear if they feel that's necessary or not.

travisleithead · 2017-11-21T01:09:16Z

@toji just so I understand what's being proposed here...

Page A: loads in UA

user clicks a button (user-initiated action) and the script requests an exclusive session.
the Promise for an exclusive session is fulfilled (because user-action requirement met). User puts on headset (optionally after first inserting device into headset, if applicable); starts enjoying really cool VR experience.
The experience offers a way to navigate to another URL, and the user "clicks" on it (while still 'in VR')

Page A: unloads

Existing VR session stops (goes black in headset or switches to 'home' environment, etc.--which would be interesting for a Google Cardboard-like device to do)

Page B: loads

While page B is navigating/parsing HTML/etc, the user either sees:
- nothing (black)
- continuation of previous 'home' environment (the 'in between' VR experience)

If Page B is NON-VR (not what this PR is about, but still interesting to consider)

User either:
1. Keeps seeing black because the UA is either hiding the regular 2D content while the device is still mounted in the headset [Cardboard-like case] or is no longer being powered by a VR experience [VR headset attached to PC case]
2. continues in the 'home' environment
  1. Home environment is being used in place of a black screen [Cardboard-like case] until the user removes the device from headset.
  2. Home environment is where the user started browsing 2d content to begin with [HoloLens-like case]. Can just resume browsing the 2d content within 3d.
  3. Home environment is default experience when no VR exclusive mode is active [VR headset attached to PC case]. User must take off headset to browse further
3. Sees the 2D page
  1. In the Cardboard-like case, the page will be much too close to the eyes, or perhaps presented mirrored 50/50 to each eye?
  2. For an attached headset to a PC, it's conceivable that the UA can similarly present the 2D web content to the headset in stereo...

If Page B is VR-capable -- what this PR is try to solve...

To avoid "falling out of VR" (because of the user-interaction requirement to start a session) resulting in the user swapping (temporarily) to one of the previously mentioned 2d experience, the idea is to continue a VR experience by re-engaging Page B's experience ASAP while the user still has the headset in place.

The proposal is to have a flag that is passed to requestSession (waitForNavigate: true) by Page B, which enables it to bypass the requirement that there be a user-initiated action to 'enter VR'. Did I get that correct?

Surely this can't be the only signal, since in that case Page A would also use this and try to bypass the user-interaction requirement which was put in place for good reason.

It seems totally reasonable to me that the UA could resume the VR experience for Page B automatically, and does not need this programmatic signal from the web developer to enable this scenario to work. The UA already has all the state info it needs to make this decision (e.g., is the headset still 'activated', receives the request to requestSession in some reasonable time) and the developer signal offers nothing new AFAICT.

Given the very unpredictable latency involved (and load times) of navigation, the user might be left waiting for quite a while before a requestSession call is made--and I would expect UAs to take reasonable steps to swap to a 'home environment' if there was a long enough gap between when the previous experience ended and the new one was requested--which is work they might chose to do anyway to have a nice experience in the navigate to a non-VR-continuation experience. I also wouldn't expect a change in the API to make this possible.

Thanks for letting me brain-dump here.

toji · 2017-12-06T01:07:32Z

Thanks for your comments, @travisleithead! In general I would say your assessment is correct, though we would likely treat Cardboard differently than you indicated. On Google's side I would expect the transitions to look something like this:

Cardboard & Daydream, VR-to-VR:

On navigation page fades out, user sees a transition state (either black or some loading indicator/spinner/minimal environment)
During page load the requestSession call is made, succeeds.
New page VR content fades in

Daydream, VR-to-2D:

On navigation page fades out, user sees transition state
requestSession is not called in time.
Transition back to standard browsing in VR, which shows the 2D page as a quad in space.

Cardboard, VR-to-2D:

On navigation page fades out, user sees transition state
requestSession is not called in time.
Message is shown to the user indicating they should remove the phone from the headset to continue browsing in 2D.

We haven't determined our preferred flow for desktop headsets yet.

Regarding the flag being seen as unnecessary, I understand what you and Nell are getting at and I agree that in this isolated example it's unnecessary. I do have concerns about how that behavior will mix with other desired flags in the future, though. For example, even though it's been removed from the patch for now I am still planning on adding the activate feature in a future patch. (It's an important part of the Daydream UX, if nothing else.) There an explicit flag must be used because there's no implicit timing under which an activation event would occur. So consider this: I have a page which for some reason I want to have start an exclusive session on activation but not on navigation. The theoretical call is:

device.requestSession({ exclusive: true, waitForActivate: true });

However, if I call that before onLoad is finished firing does that count as asking for navigate behavior as well? Does the user delay making that call until after onLoad has fired or does waitForActivate negate the navigation behavior. If so that would appear to make the solution for waiting on either navigate or activate the following:

device.requestSession({ exclusive: true });
device.requestSession({ exclusive: true, waitForActivate: true });

Which certainly works, but feels strange to me. I find the explicit variant much easier to understand regarding developer intent:

device.requestSession({ exclusive: true, waitForNavigate: true, waitForActivate: true });

That's really just syntactic sugar at the end of the day, and I wouldn't consider it a blocker. Still, I'm curious if it changes your opinion.

toji · 2017-12-06T21:19:49Z

Oh, one other benefit of having an explicit flag: It allows us to communicate more useful error messages to the developer calls it outside the page load window.

Attempted to handle VR navigation too late in the page lifetime.

instead of

Unable to request a session outside of a user gesture.

toji · 2018-02-20T18:47:00Z

Rebased this (surprisingly old) pull request and renamed all the VR things to XR. Didn't change the basic substance, though, because there was never a followup to my previous comments.

Would like to discuss merging this on today's call.

Most comments addressed, minus ongoing waitForNavigate discussion

ddorwin · 2018-02-20T19:11:26Z

explainer.md

+
+WebXR applications can, like any web page, link to other pages. In the context of an exclusive `XRSession`, this is handled by setting `window.location` to the desired URL when the user performs some action. If the page being linked to is not XR-capable the user will either have to remove the XR device to view it or the page could be shown as a 2D page in a XR browser.
+
+If the page being navigated to is XR capable, however, it's frequently desirable to allow the developer to immediately create an exclusive `XRSession` for that page as well, so that the user feels as though they are navigating through a single continuous XR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "XR navigable".


The last two sentences are a bit confusing. I think being more explicit and consistent about the actors would help.

ddorwin · 2018-02-20T19:13:38Z

explainer.md

+
+If the page being navigated to is XR capable, however, it's frequently desirable to allow the developer to immediately create an exclusive `XRSession` for that page as well, so that the user feels as though they are navigating through a single continuous XR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "XR navigable".
+
+In order to start presenting automatically when a page is XR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is XR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not XR navigable the request will reject immediately.


the request will be resolved when the page is ready to begin presenting.

"The page" doesn't determine when it is ready. The UA must make this decision based on something. We should specify what that is. This is especially important to analyze the lifetime impacts and what applications can expect.

It's probably also worth (non-normatively) addressing what pages should do to ensure a smooth transition. For example, display frames quickly and defer work to post-presentation. This is important because the user experience may be different from a page that normally loads a magic window and has loaded the resources before calling requestSession().

ddorwin · 2018-02-20T19:18:03Z

explainer.md

+
+If the page being navigated to is XR capable, however, it's frequently desirable to allow the developer to immediately create an exclusive `XRSession` for that page as well, so that the user feels as though they are navigating through a single continuous XR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "XR navigable".
+
+In order to start presenting automatically when a page is XR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is XR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not XR navigable the request will reject immediately.


"deferred session" is introduced as a term here and used below.

The session isn't really deferred. I think it's the request that is deferred, which is consistent with "deferred" session request.

If a term is going to be coined, maybe it should be included in the section header. For example, "Page navigation & deferred session requests."

ddorwin · 2018-02-20T19:18:29Z

explainer.md

+
+If the page being navigated to is XR capable, however, it's frequently desirable to allow the developer to immediately create an exclusive `XRSession` for that page as well, so that the user feels as though they are navigating through a single continuous XR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "XR navigable".
+
+In order to start presenting automatically when a page is XR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is XR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not XR navigable the request will reject immediately.


nit: comma after the last XR navigable.

ddorwin · 2018-02-20T19:18:54Z

explainer.md

+
+In order to start presenting automatically when a page is XR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is XR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not XR navigable the request will reject immediately.
+
+Deferred sessions must be exclusive. If a non-exclusive deferred session is requested the promise will immediately be rejected. Deferred requests do not need to be made within a user gesture. Only one session request with the `waitForNavigate` option is allowed per page and subsequent requests using `waitForNavigate` will be rejected immediately. The page's XR navigable state must expire after all listeners for the `window`'s `load` event have fired.


nit: comma after requested.

ddorwin · 2018-02-20T19:44:57Z

explainer.md

+
+WebXR applications can, like any web page, link to other pages. In the context of an exclusive `XRSession`, this is handled by setting `window.location` to the desired URL when the user performs some action. If the page being linked to is not XR-capable the user will either have to remove the XR device to view it or the page could be shown as a 2D page in a XR browser.
+
+If the page being navigated to is XR capable, however, it's frequently desirable to allow the developer to immediately create an exclusive `XRSession` for that page as well, so that the user feels as though they are navigating through a single continuous XR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "XR navigable".


"XR navigable" sounds like a property of the page, but this is really state within the UA. I think we should find another term that reflects "allowed to present on navigation."

ddorwin · 2018-02-20T19:48:44Z

explainer.md

+
+WebXR applications can, like any web page, link to other pages. In the context of an exclusive `XRSession`, this is handled by setting `window.location` to the desired URL when the user performs some action. If the page being linked to is not XR-capable the user will either have to remove the XR device to view it or the page could be shown as a 2D page in a XR browser.
+
+If the page being navigated to is XR capable, however, it's frequently desirable to allow the developer to immediately create an exclusive `XRSession` for that page as well, so that the user feels as though they are navigating through a single continuous XR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "XR navigable".


If we keep using a term like "XR navigable," I suggest italicizing it in subsequent uses or otherwise making it clear that this is a special state.

ddorwin · 2018-02-20T19:54:05Z

explainer.md

@@ -666,6 +689,7 @@ partial interface Navigator {

 dictionary XRSessionCreationOptions {
  boolean exclusive = false;
+  boolean waitForNavigate = false;


waitForNavigate is an odd name since the navigate has already occurred by the time the page starts running code, right?

We should probably use a term related to whatever it means for the UA/page to be "ready to begin presenting."

ddorwin · 2018-02-20T20:03:18Z

explainer.md

@@ -605,6 +605,29 @@ xrSession.addEventListener('resetpose', xrSessionEvent => {
 });
 ```

+### Page navigation
+


This section doesn't address how the user agent behaves before/during the navigation other than to track "when a page navigates without ending an active exclusive session first."

Specifically, the UA needs to know whether to remain in XR presentation or transition to 2D. As written, I think the UA would always need to remain in XR presentation until "after all listeners for the window's load event have fired." That would be different from navigating from all other pages. We might need some type of HTTP header to trigger this behavior. We should talk to navigation folks.

ddorwin · 2018-02-20T20:32:39Z

explainer.md

@@ -605,6 +605,29 @@ xrSession.addEventListener('resetpose', xrSessionEvent => {
 });
 ```

+### Page navigation
+


I see that headset activation was removed. While it doesn't have to be in this PR, I think we should know how we want to handle that (i.e., have a companion PR) before we effectively commit to an API shape for "auto presentation" scenarios.

ghost · 2018-04-06T08:41:05Z

Sorry to revive this issue, but I didn't see any information about the origin URL from where the deferred session has been called. How the target website can know this information, to create a "go back" portal by example?

toji · 2018-04-06T18:19:55Z

This issue needs reviving. :)

The origin URL should be handled by the existing document.referrer attribute.

ghost · 2018-04-09T08:18:32Z

So yeah I was wondering if document.referrer was "strong enough" to be used. By example, clicking in the email I received from your answer to this issue, the referrer is empty. Isn't there a risk that'll happen as well through link traversal?

Also, I've some concerns:

should the UA manage "go back" to prevent misbehavior? (false redirection, trapping the user)
should the UA offer a way to preview an experience through a portal? (access an equirec pano or something in the liking)

And finally I was wondering if we could identify WebXR websites in an automated way, through some meta tag or something, to allow indexation through some search portal.

Also it's the main issue I found about navigating from one experience to an other, but maybe some of those question belong to new issues. If so please let me know, I'd be happy to create new ones!

blairmacintyre · 2018-04-09T10:56:59Z

I would tend to assume, given the myriad issues around user privacy and the web, that we should stick to the semantics of document.referrer and not try to reinvent "when is it ok to tell a page what the referring page is".

Don't UAs already have a mechanism to "go back"? Shouldn't we just use that?

As for the "preview portal" we've worked on demos of that in the past (@cvan?) , and it should definitely be possible. We don't really want to download and run entire pages to generate the world through the portal, so this may require something like meta tags, glTF preview models, etc.

jsantell · 2018-05-07T15:07:54Z

We should be sure that the initiation of any deferred sessions carry with them sufficient privileges/"engagement" to enable audio/video, such that it does not require additional gestures.

Context: Chrome's autoplay policy change in m66. The MEI scoring here may already be satiated, but a use-case that should be supported by deferred sessions.

Replaces #256 and compliments #382

toji · 2018-07-21T03:55:28Z

Closing this as there's now a new proposal in #383

toji commented Jul 11, 2017

View reviewed changes

toji force-pushed the activate-promise branch from 689835d to f5f3452 Compare July 11, 2017 22:41

toji mentioned this pull request Jul 21, 2017

Remove the navigation event from the explainer #260

Merged

toji force-pushed the activate-promise branch from 3a06913 to 5045aa0 Compare July 22, 2017 05:15

toji force-pushed the activate-promise branch from 5045aa0 to 5adae63 Compare September 21, 2017 15:59

toji force-pushed the activate-promise branch 3 times, most recently from 4096897 to c228adb Compare November 1, 2017 19:15

cvan previously requested changes Nov 3, 2017

View reviewed changes

cvan mentioned this pull request Nov 4, 2017

Do not treat activate events as a user gesture #248

Closed

NellWaliczek previously requested changes Nov 13, 2017

View reviewed changes

cvan mentioned this pull request Nov 25, 2017

Linking between VR web pages #30

Closed

cvan mentioned this pull request Dec 14, 2017

Should the Feature Policy feature name be changed from "vr"? #308

Closed

toji force-pushed the activate-promise branch from 16f7fb1 to 1c7572f Compare February 20, 2018 18:36

ddorwin reviewed Feb 20, 2018

View reviewed changes

ddorwin mentioned this pull request Mar 20, 2018

Revisit session creation options and flow #330

Closed

toji mentioned this pull request Mar 26, 2018

Consider using AbortController/AbortSignal instead of cancelRequestAnimationFrame #335

Closed

Introduce concept of deferred session requests

9c69862

toji force-pushed the activate-promise branch from 06bd700 to 9c69862 Compare April 2, 2018 23:15

Partially addressing @ddorwin's comments.

ebb744b

toji added a commit that referenced this pull request Jul 20, 2018

Added a section on immersive navigation to the explainer

64e8560

Replaces #256 and compliments #382

toji mentioned this pull request Jul 20, 2018

Immersive navigation proposal #383

Closed

toji closed this Jul 21, 2018

toji deleted the activate-promise branch July 21, 2018 03:55


		Many VR devices have some way of detecting when the user has put the headset on or is otherwise trying to use the hardware. For example: an Oculus Rift or Vive have proximity sensors that indicate when the headset is being worn. And a Daydream device uses NFC tags to inform the phone when it's been placed in a headset. This is referred to as the `VRDevice` being "activated", and depending on context it may represent the user showing a clear intent to begin using VR. Many WebVR application may want to begin presenting automatically in these scenarios.

		In order to start presenting when the `VRDevice` is activated pages can request a deferred session. To request a deferred session the option `{ waitForActivate: true }` is passed into the `requestSession` call. When a session is requested with `waitForActivate` the request will remain outstanding until the `VRDevice` is activated, at which point the promise will resolve with the requested session (or reject, if necessary.)


		In order to start presenting when the `VRDevice` is activated pages can request a deferred session. To request a deferred session the option `{ waitForActivate: true }` is passed into the `requestSession` call. When a session is requested with `waitForActivate` the request will remain outstanding until the `VRDevice` is activated, at which point the promise will resolve with the requested session (or reject, if necessary.)

		Deferred sessions must be exclusive and will not be fulfilled if there is already an exclusive session active for the VR hardware device when the activation action occurs. Deferred requests do not need to be made within a user gesture.


		Once the deferred session request has been fulfilled or rejected a new deferred session request will need to be issued if the page wishes to respond to future activation actions.

		To detect when the user removes the headset, at which point the page may want to end the session, listen for the `deactivate` event. Note that not all devices capable of detecting activation can reliable detect deactivation, so pages are not guaranteed to receive an associated deactivate event for each activation action.


		WebVR applications can, like any web page, link to other pages. In the context of an exclusive `VRSession`, this is handled by setting `window.location` to the desired URL when the user performs some action. If the page being linked to is not VR-capable the user will either have to remove the VR device to view it or the page could be shown as a 2D page in a VR browser.

		If the page being navigated to is VR capable, however, it's frequently desirable to allow the user to immediately create an exclusive `VRSession` for that page as well, so that the user feels as though they are navigating through a single continuous VR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "VR navigable", which activates a couple of new behaviors to allow for the desired navigation experience. UAs may also place additional restrictions on what conditions are considered VR navigable, such as restricting it to only same-origin URLs.


		If the page being navigated to is VR capable, however, it's frequently desirable to allow the user to immediately create an exclusive `VRSession` for that page as well, so that the user feels as though they are navigating through a single continuous VR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "VR navigable", which activates a couple of new behaviors to allow for the desired navigation experience. UAs may also place additional restrictions on what conditions are considered VR navigable, such as restricting it to only same-origin URLs.

		In order to start presenting automatically when a page is VR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is VR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not VR navigable the request will reject immediately.


		WebXR applications can, like any web page, link to other pages. In the context of an exclusive `XRSession`, this is handled by setting `window.location` to the desired URL when the user performs some action. If the page being linked to is not XR-capable the user will either have to remove the XR device to view it or the page could be shown as a 2D page in a XR browser.

		If the page being navigated to is XR capable, however, it's frequently desirable to allow the developer to immediately create an exclusive `XRSession` for that page as well, so that the user feels as though they are navigating through a single continuous XR experience. To allow this the UA tracks when a page navigates without ending an active exclusive session first. This flags the newly loaded page as "XR navigable".


		In order to start presenting automatically when a page is XR navigable, pages can make a "deferred" session request. To request a deferred session the option `{ waitForNavigate: true }` is passed into the `requestSession` call. If the page is XR navigable, the request will be resolved when the page is ready to begin presenting. If the page is not XR navigable the request will reject immediately.

		Deferred sessions must be exclusive. If a non-exclusive deferred session is requested the promise will immediately be rejected. Deferred requests do not need to be made within a user gesture. Only one session request with the `waitForNavigate` option is allowed per page and subsequent requests using `waitForNavigate` will be rejected immediately. The page's XR navigable state must expire after all listeners for the `window`'s `load` event have fired.

Deferred session requests #256

Deferred session requests #256

Conversation

toji commented Jul 11, 2017 • edited by pr-preview bot Loading

Choose a reason for hiding this comment

toji commented Jul 11, 2017

RafaelCintron commented Jul 27, 2017 • edited Loading

toji commented Jul 27, 2017

RafaelCintron commented Jul 27, 2017

toji commented Sep 26, 2017

Brandon's Proposal

Rafael's Proposal

kearwood commented Sep 26, 2017

toji commented Sep 29, 2017

cvan commented Oct 24, 2017

toji commented Nov 1, 2017

toji commented Nov 2, 2017

toji commented Nov 2, 2017

cvan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toji commented Nov 3, 2017

cvan commented Nov 3, 2017

toji commented Nov 4, 2017

toji commented Nov 11, 2017

toji commented Nov 13, 2017

NellWaliczek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toji commented Nov 14, 2017

travisleithead commented Nov 21, 2017

toji commented Dec 6, 2017

toji commented Dec 6, 2017

toji commented Feb 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Apr 6, 2018

toji commented Apr 6, 2018

ghost commented Apr 9, 2018

blairmacintyre commented Apr 9, 2018

jsantell commented May 7, 2018

toji commented Jul 21, 2018

toji commented Jul 11, 2017 •

edited by pr-preview bot

Loading

RafaelCintron commented Jul 27, 2017 •

edited

Loading